Participation and engagement in online cognitive testing

Web-based testing of cognitive abilities allows for large-scale assessments without geographical constraints. Yet, the extent to which it can reach populations beyond the typical demographic groups recruited for cognitive studies is unclear. This study focused on comparing the characteristics of individuals from a general population study (HUNT4) who chose to participate in a cognitive study (HUNT4-Hjernetrim) with those who did not. Additionally, we investigated participants' engagement and user experience. We obtained data on socio-demographics, health conditions (both physical and mental), self-reported cognitive or learning difficulties, and lifestyle factors of Hjernetrim participants and non-participants from the HUNT4 database. Hjernetrim involved 13 cognitive tests, administered through the online platform Memoro. We used logistic regressions to assess participation biases and linear regressions to assess participants’ engagement and user experience. Of 65,851 HUNT4 participants invited via regular mail to Hjernetrim, 5634 (9.4%, aged 13–97, 54% women) participated. The best represented in the sample were 50–79-year-olds, women, tertiary educated, living alone, from urban areas, not occupationally active, and reporting memory complaints. Individuals who were aged 80+, had motor or vision impairments, and teenagers with learning disabilities, were underrepresented. Participants were more likely to have mental health problems, have or survived cancer and less likely to have cardiovascular disease. Participants logged on mainly during weekdays, the preferred time of day varied by age. On average, participants used 42 min and completed 78% of the tasks. Using PCs provided the most complete data. In terms of user experiences, 65% were positive while 14% were negative or reported technical difficulties. Overall, the study demonstrated that web-based methodology allowed for a relatively well-represented sample that included groups typically difficult to reach. The presence of somatic and mental diseases had a variable influence on participation. Participants finished most tests and reported positive experiences overall.


Study population and procedure
The HUNT4 study The HUNT4 study is the fourth wave of the Trøndelag Health Study, one of the world's largest ongoing longitudinal population-based health studies 13 .The HUNT study is considered representative of Norway geographically and economically with age, morbidity, and mortality distribution similar to the national averages.HUNT4 data collection was approved by the Norwegian Data Inspectorate and Regional Committee for Research Ethics (REK-2016/801).Informed consent was obtained from all subjects and/or their legal guardian(s) and the study was performed in accordance with relevant guidelines and regulations, including the Declaration of Helsinki.
The research sample consisted of three geographical and age-based cohorts: the HUNT4-N (NT4) cohort consisted of all residents aged 20 years and older in the northern part of Trøndelag County, the Young-HUNT4 (YH4) cohort consisted of residents of the same area between ages 13 and 19, and finally the ST1 cohort consisted of adults aged 70 years and older living in three districts of Trondheim, the largest city in the region.
The three cohorts filled out mostly overlapping cohort-specific questionnaires, where some questions were adapted to the age group (e.g., adults were asked about cognitive issues while the youth were asked about learning disabilities).From the HUNT4 study the following variables were acquired: age (calculated from the date of birth to the day of invitation to NT4, YH4, or ST1), biological sex, education level, living alone or with a partner, work status, place of residence (rural vs. urban), general health issues and specific diagnoses, including scores on the Hospital Anxiety and Depression Scale (HADS), self-reported cognitive or learning problems, lifestyle satisfaction, and current smoking and alcohol use (Table 1).The scoring of the HADS was categorized into two groups: (1) scores below were considered within the normal range; (2) scores of 11 or greater indicated a potential clinical range of symptoms of anxiety and depression (fhi.no/globalassets/dokumenterfiler/rapporter/2016/maleegenskaper-ved-den-norske-versjonen-pdf.pdf).A detailed description of all variables included in the study can be found on the HUNT Databank website (hunt-db.medisin.ntnu.no/hunt-db).
In addition, the HUNT Databank includes data on age and sex from the Norwegian National Registry, on all those residing in the catchment area where the HUNT4 study was performed, i.e., also those who did not participate in HUNT4, which we obtained.

The HUNT4 Hjernetrim study
The HUNT4 Hjernetrim Study (hereafter called Hjernetrim) was accepted as a HUNT4 substudy by the HUNT4 planning committee in 2015.The data collection was approved by the Norwegian data inspectorate study as part of HUNT4, the Regional Committee for Research Ethics (REK-155024 HUNT4 Hjernetrim study), and the HUNT administration.The participants gave their informed consent online before the testing started.The study was performed in accordance with relevant guidelines and regulations, including the Declaration of Helsinki.
As required by the HUNT Research Centre, only participants who had taken part in the main HUNT4 study could be invited to Hjernetrim.An invitation letter for the Hjernetrim study was included in the feedback letter and sent by regular mail to participants from the three cohorts (NT4, YH4, ST1), totalling 65,851 invited individuals (54% women), of whom 5634 participated (54% women).The letter contained the results of the clinical assessment and physical activity accelerometer results collected in HUNT4 and personal health recommendations.The letter also included a half-fold 10.5 cm by 14.5 cm leaflet providing a brief description of the Hjernetrim study, its website address, and the participant-specific login credentials required to log on to the website.Participants were given instructions to manually enter this web address and their login credentials into their web browsers to access the web-based cognitive test platform, Memoro, for participation.Invitation letters were sent out continuously by regular mail as the results from the HUNT4 data collection were processed.The possibility of participating in Hjernetrim was additionally advertised in local newspapers, by flyers at shopping malls, and at the HUNT4 field stations.
Participants were told to set aside about 40 min to complete the tests, be in a quiet place, and preferably use a personal computer with the Google Chrome browser, although other platforms and browsers were also supported.
The Hjernetrim participation was conducted using our proprietary, validated, self-administered, web-based cognitive test platform Memoro 3,14 to perform a selection of tests covering several domains.A total of 13 tests were administered in a fixed order: Simple Reaction Time, Pattern Separation, Visual Memory (Immediate Recall), Verbal Memory (Learning and Immediate Recall), Symbol-Digit Coding (Main Test), Symbol-Digit Coding (Recall of a Symbol-Digit Key), Digit Span Forward, Visual Memory (Delayed), Visual Memory (Recognition), Digit Span Backwards, Verbal Memory (Delayed Recall), Verbal Memory (Recognition), Complex Reaction Time.
Before cognitive testing started, the participants consented online and filled in a short questionnaire including birth year, gender, level of education, handedness, and computer familiarity assessed on a 5-point Likert scale ("How comfortable are you with computers?").We used gender, age in years, and level of education (three levels, see above) to calculate normalized scores for participant-specific feedback on performance on the battery (see below).The instructions for tasks were both written and verbally presented.The main tasks were preceded by short training sessions which participants needed to perform correctly in order to proceed to the main tasks.If training results indicated that task instructions were not understood, participants were automatically transferred to the next test.
For participants not familiar with using computers or not owning one, Hjernetrim field stations were arranged and advertised together with Hjernetrim information.Those who chose to participate in the field stations could use the computer equipment and technical assistance of the researchers available on-site and could log in using either their original credentials or, to keep the subject anonymous, use temporary credentials which were later connected back to their HUNT ID.
Participants who completed at least one test could access a feedback page that was available after completing or aborting the last task.Participants needed to log in anew to access the feedback.The mode of feedback was developed in collaboration with the HUNT administration and their user group before data collection started.The feedback page displayed details regarding the cognitive skills evaluated in Memoro, along with practical examples of how these abilities are employed in daily life.Participants could see how well they performed on the different tasks in comparison to others of the same gender, age, and education group rated from the participant's best to the worst test performance and for each test whether the performance was above, at, or below the participant's age, gender and education group average.

Engagement, user experience, and feedback
We collected metadata on Hjernetrim participation for each log-in.The metadata included device type, operating system, web browser type and version, number of logins, the date and time of starting and finishing each of the tasks, and the status of each task (started, finished, aborted, failed).
For participants logging more than once and using multiple devices, we defined the participant's main device and web browser engine as the ones that were used to complete the highest number of tasks.For participants who completed an equal number of tasks on more than one device and/or web browser, we used the one that was used first.
The time and day of participation were registered at the first login to Memoro.To analyze them, we defined one day as the period between 4 and 4 am of the next day to separate the "night owls" from the "early birds".If a participant completed the battery in more than one session, only the first session was considered.
We calculated battery completion time as the sum of the time taken to complete the training and main tasks from start to finish, measured in minutes, excluding the breaks that participants took between the tasks.Tasks in which participants spent less than 5 times the interquartile range (IQR) or more than 10 times the IQR were considered outliers and excluded from the analysis.
We defined the number of finished tasks as the sum of main tasks that were not aborted by participants or as a result of failed training, and that provided valid scores.Training tasks were not included in the battery completion statistics.
Participants who did not start all tasks were categorized as having discontinued the battery.Successful completion of all 13 tasks was not necessary to avoid being classified as having discontinued; participants only needed to initiate each task.
After the testing session concluded, participants were asked to indicate whether they had encountered any disruptions or interruptions during one or more tasks.They could do this by marking the affected tasks off on a provided list.They were also asked to share their overall experience with the testing in the form of an open question "Please, tell us about your experience".The free text responses were subsequently stratified into the following categories: general positive experience (e.g."Fun"); general negative experience (e.g."Stressful"); comments about battery difficulty or own performance (e.g."Struggled with word task, was better with numbers"); any technical issues with the platform (e.g."Aborted a task by mistake", "Sound problems", "Images did not appear"); and other comments (typically about a specific task, like its length, instructions, or about being interrupted, e.g."Disturbed by dad ", "Difficult, especially with a screaming baby").
Contact information for the researchers (e-mail address and phone number) was available in the invitation mail and on the Memoro website for participants who experienced technical issues, wanted to participate in a field station, or had other inquiries.

Statistical analysis
Participants' sociodemographic and health characteristics, as well as metadata from the web testing, are presented as frequencies and percentages or means and standard deviations, as appropriate.

Participation bias
To examine factors affecting the likelihood of participation in Hjernetrim, we compared those participating in HUNT4 but not in Hjernetrim (n = 60,217) and those participating in both studies (n = 5634) as the dependent variable in logistic regression.For each variable, the group with the largest number was used as the reference group when reporting the odds ratios (OR).First, we examined age groups in decades, sex, and education (stratified into primary, secondary, and tertiary).In all subsequent logistic regression models, age group, sex, and education were included as variables of no interest.We investigated other sociodemographic variables next, namely living alone or with a partner, work status, and place of residence (urban or rural).Subsequently, we investigated health-related variables: general health and impairment(s), and the presence of specific somatic and www.nature.com/scientificreports/mental diagnoses/problems.Then we investigated the impact of cognitive and learning issues.Next, the impacts of satisfaction with own lifestyle, smoking, and alcohol use were assessed.Finally, as age and sex were available for participants from all invited to HUNT4 (n = 119,558) including those who did not participate and thus were not invited to the Hjernetrim study, we used it to assess the effects of age and sex on the likelihood of participation in Hjernetrim among the general population.We used a logistic regression model with participation as a dependent variable and age group and biological sex as variables of interest.

Engagement in Hjernetrim
To assess engagement among those participating in HUNT4 Hjernetrim, we used linear regression models with the time of day and weekday of starting the battery, time spent in the battery, and number of finished tasks as the dependent variable in separate models with age or age group, sex, education, and, where appropriate, device used and computer familiarity, as independent variables.
To assess factors leading to discontinuation and leaving feedback, we used logistic regressions with discontinuation and leaving feedback as the dependent variable in separate models.In both models, the independent variables were age group, sex, education level, device used, and computer familiarity.For the model analyzing leaving feedback, we additionally included the number of tasks completed by the participants as an independent variable.
To assess the level of computer familiarity across Hjernetrim participants, we used ordered logistic regression with computer familiarity as a dependent variable and age group, sex, and education as independent variables.
For dummy variables, the most numerous subgroup from HUNT4 for each variable was used as the reference group when reporting odds ratios (OR).

Sample characteristics
Hjernetrim data were collected during the HUNT4 study between November 2017 and February 2021, with most of the data (69%) collected in 2018 (Fig. 1a).Overall, 5,634 participants (3,254 women, 2,380 men, 13 to 97 years old, Fig. 1b) gave consent to participate in Hjernetrim, which translates into 8.6% of those invited and 4.7% of the general population (Fig. 2).An overview of the sociodemographic, health, cognitive, and lifestyle-related variables in the HUNT4-only participants and those who also participated in Hjernetrim are presented in Table 2.

Lifestyle satisfaction, smoking, and alcohol use
Compared to those who were satisfied with their lifestyle (diet, exercise, smoking and drinking habits), those who were very satisfied were more likely to participate (OR = 1.17, 95% CI [1.06, 1.28]) (Fig. 5).Compared to those who never smoked, ex-occasional smokers were more likely (OR = 1.

Demographics of Hjernetrim participants compared to the general population
This analysis compared the distribution of age and sex in those who participated in Hjernetrim and everybody else in the catchment area, i.e., both those participating in the HUNT4 study and not.These results were quite consistent with the above results, showing the 60-69-year group to be the most likely (OR = 1.20, 95% CI [1.11, 1.31]), and the ≥ 80-year-olds to be the least likely (OR = 0.20, 95% CI [0.16, 0.24]) to participate in Hjernetrim.
Men were less likely to participate (OR = 0.71, 95% CI [0.67, 0.75]) than women.The distribution of age and sex among Hjernetrim participants and those in the catchment area who did not participate is presented in Supplementary Table S1.The full results of logistic regression are presented in Supplementary Table S2.

Hjernetrim engagement
Computer familiarity, hardware, and software The majority of Hjernetrim participants rated themselves as familiar with computers (Table 2, Supplementary Table S3), with 90.6% of men and 89.2% of women reporting being rather comfortable or very comfortable using them.Being younger, male, and having more education were associated with higher computer familiarity (Supplementary Table S4).
The mean time of starting the battery was 14:10, with 90% of participants starting between 7:33 and 20:42.Preferred starting times varied widely across different age groups.The data showed that younger participants generally started in the afternoon, those in working age tended to start in the evenings, while older adults preferred starting in the mornings (Fig. 7).

Time spent
For participants who completed all the tasks, the mean duration was 47 min, with a median of 46 min and a range from 27.6 to 96.7 min (Fig. 8a).The time to complete the whole battery increased linearly with age (t = 18.35, p < 0.001), with a mean increase of time of about 1 min per 7 years of age.When including participants who discontinued, the average time spent was 42 min, with a range from 0.2 to 96.7 min and a median of 44 min (Fig. 8b).

Completion and discontinuation of test battery
The completion rate was high with 58% (n = 3248) completing all tasks and an additional 22.5% (n = 1270) completed 7 tasks or more.A total of 13.2% (n = 743) participants completed < 7 tasks, while 6.7% (n = 373) did not complete any tasks.The mean number of completed tasks was 10.3 out of 13 tasks.Several factors influenced task completion (R 2 = 0.06, F(16, 5313) = 21.28,p < 0.001), including education, age, and device (Table 3).Participants with tertiary education were more likely to complete all tasks compared to those with secondary education.The number of completed tasks was lower in older adults.Sex and computer familiarity were not related to the number of tasks completed, but using an Android device, an iPhone or an iPad negatively influenced completion rates (Fig. 9).
Likewise, those with tertiary education were less likely to discontinue the battery (OR = 0.72, 95% CI [0.59, 0.88]), while participants who used Android devices (OR = 1.63, 95% CI [1.10, 2.11]) and iPhones were more likely to discontinue (OR = 2.17, 95% CI [1.49, 3.17]).Sex and computer familiarity were not related to discontinuation.Odds Ratio (95% CI) Figure 3.The odds ratio of age group, sex, and education on participation in Hjernetrim relative to participation in HUNT4.Fifty-year-olds, women, and those with secondary education were used as references as these were the most numerous subgroups.For details on variables and their frequency, see Tables 1 and 2.

Figure 4.
The influence of self-reported somatic and mental health problems, as well as learning disabilities and self-reported memory problems in HUNT4 on the probability of participating in Hjernetrim.Each color represents a separate model.Fifty-year-olds, women, and those with secondary education were used as references as these were the most numerous subgroups.For details on variables and their frequency, see Tables 1  and 2.
Vol:.( 1234567890) The test associated with this greatest discontinuation was the learning trials of the Verbal Memory task (n = 249, corresponding to 41% of all such cases and 4% of all participants).
Participation at field stations Overall, 23 (0.4%) participants performed the battery in field stations.Their mean age was 73 years (49.9-85.6 years), which is significantly higher than the rest of the sample (t = 6.06, p < 0.001).Those participants completed fewer tasks on average compared to the entire sample (8.6 vs 10.3), but the difference was not statistically significant (t = − 1.07, p = 0.284), likely due to the low number of field station participants and large variability of the number of tasks they completed.Participants at the field stations spent an average of 52.7 min (19.5-89.9min) on Memoro tasks, over 10 min longer than those who solved them at home (t = 3.10, p = 0.002).

User experience and feedback for participants
A total of 27.6% reported being interrupted or disturbed on at least one task.A total of 58.1% answered the open question about their experience with the test battery.Participants were more likely to leave a comment if they finished all the tasks compared to those who finished most (OR = 0.65, 95% CI [0.56, 0.76]), less than half (OR = 0.006, 95% CI [0.003, 0.010]), or none of the tasks (OR < 0.001, 95% CI [< 0.001, 0.001]), and the probability increased with age between 20 and 80 years.Conversely, the group aged 13-19 was significantly more likely, while the 80 + age group was significantly less likely to provide comments on their experience compared to their neighboring age groups.Sex and education did not influence the likelihood of leaving comments.An overview of the comments stratified by topics is presented in Table 4.
Only 20.4% (n = 1147) logged back into the Memoro website to access their performance results and an additional 0.8% (n = 44) contacted us directly to receive their results.

Discussion
The study had two main objectives.The first objective was to compare the characteristics of individuals who participated in the HUNT4-Hjernetrim cognitive substudy with those who did not, focusing on sociodemographics, self-reported health, self-reported cognition, and lifestyle.The second objective was to evaluate participants' engagement with the study with regards to their computer familiarity and preferred device and browser; the time of day and week they took the test; the time spent completing the tasks; the number of tasks completed and reasons for discontinuing; participation at field stations; and their self-reported user experience and the frequency with which they accessed their performance results.
To our knowledge, this is the first study to examine participation bias and engagement in web-based cognitive testing in a general population.Our study provides insight into sociodemographic, health, self-reported cognition, and lifestyle characteristics of those opting in and completing web-based cognitive testing.The overall participation rate was only 8.6% of those invited and 4.7% of the general population, suggesting substantial room for improvement in how participants are invited, re-invited and reminded of such studies.Importantly, the field stations were an ineffective means of recruitment.Those self-selecting into web-based testing in HUNT4 were predominantly between 50 and 80 years old and had tertiary education.Web-based testing appeared to appeal more to participants living alone and not working, and those with self-reported good health, certain diagnoses (e.g., cancer, mental health) and subjective experienced cognitive issues among adults, but not to teenagers with Odds Ratio (95% CI) Figure 5.The influence of self-reported lifestyle factors in HUNT4 on the odds ratio of participating in Hjernetrim.Each color represents a separate model.Fifty-year-olds, women, and those with secondary education were used as references as these were the most numerous subgroups.For details on variables and their frequency, see Tables 1 and 2.
learning problems.To reach all age groups, web-based, unsupervised cognitive testing needs to be available on a multi-device platform.Finally, when participants did log in and engage, most completed all tasks and reported a positive experience.The number of completed tasks was lower and the risk of discontinuation was higher among older participants, those without tertiary education, and those using phones or tablets rather than PCs.Weekdays were optimal for participation.

Sociodemographics
Hjernetrim participants were between 13 and 97 years of age, illustrating that web-based testing can be used to recruit participants of all age groups.The age distribution in Hjernetrim participants was comparable to those who only participated in HUNT4, but with the highest participation rate among those aged 50-69 years, whereas participants above 80 years were relatively underrepresented in Hjernetrim.Hjernetrim participation rates were higher with age until the late sixties in women and seventies in men and lower thereafter in both sexes compared to only HUNT4 participation.This result differs from previous health-related studies, for example in heart failure RCTs and a study of self-selection for digital health research participants tended to be younger than non-participants [15][16][17] .The higher participation rate among middle-aged and older (but not oldest old) in Hjernetrim could be attributed to more leisure time at these ages 18 .Another  explanation might be that people become more invested in their cognitive health as they age, which makes them more likely to view involvement in this type of research as advantageous for themselves or their age group at large 19 .Our finding of a lower participation rate among those who reported having no memory problems supports this notion.Nevertheless, the least likely to participate in Hjernetrim were those aged 80 years and above.This might be due to the general decrease in engagement in leisure activities despite more free time, observed in the oldest old 20 , which could have affected Hjernetrim more than the main study because of the latter's somatic health The odds ratios for participation were impacted by different health conditions in varying ways.The lack of influence of diabetes, stroke, hearing impairment, and HADS score over 11 on participation is probably important to note as it shows that somatic and mental health challenges do not necessarily impair the ability to take part in web-based cognitive testing.This is further exemplified by the increased participation odds in those reporting cancer, or mental health problems they sought help for.Those conditions have been connected to cognitive problems or complaints [39][40][41][42] .One could speculate that facing such challenges would make these groups more conscious about their cognitive health and thus more likely to participate, suggesting that web-based testing is a suitable method for studies targeting those groups, although previous studies showed otherwise 17 .The higher participation rates among those with mental health problems they sought help for, provide support for this reasoning.On the other hand, we observed lower participation among groups of individuals who reported "not very good" health or experienced impairment due to chronic illness or injury, myocardial infarction, or motor and vision impairments.Depending on the type of injury or chronic disease, participants could need physical therapy or treatments, and thus not have enough time or possibility to participate in this study.Likewise, participants with motor or vision impairment could have difficulties using the devices or perform the tasks.Implementation of universal design will be important to meet the needs of those with sensory and motor deficits in the future.A shorter test battery might have been better as it would reduce the overall load on participants and facilitate their participation.
Overall, in a study targeting the general population, no clear pattern emerged suggesting that diseases or impairments necessarily lead to reduced participation.Instead, the data suggest a more complex picture, with some diseases, disorders or impairments associated with higher participation rates and others associated with lower participation rates, similar to a previous study examining HUNT participation 43 .Additionally, some groups previously considered hard to reach, such as those living alone 44 and those not occupationally active appear to have a high participation rate in a study with a web-based design.

Self-reported cognition
Teenagers with learning disabilities, especially in mathematics, participated less often.Since learning difficulties negatively impact academic well-being 45 , students with learning disabilities could have felt discouraged from participating in cognitive testing.On the other hand, adults who self-reported having no memory problems were less likely to participate, which again could indicate that worrying about one's health increases participation odds.It should be noted that subjective memory complaints in this cohort were connected with lower scores on the verbal list learning test 46 , indicating correspondence between subjective and objective measures of cognition.The dichotomy in participation between teenagers and adults with self-reported cognitive issues indicates that different strategies might be needed when recruiting different age groups to cognitive testing.

Lifestyle satisfaction, smoking, and alcohol use
Being very satisfied with one´s lifestyle (diet, exercise, smoking and drinking habits) increased participation.This is a largely expected outcome given that those who tend to lead a healthier lifestyle and are satisfied with life show other characteristics associated with higher participation in our study, such as higher education, urban place of residence, and less or no chronic diseases 47 .
Individuals who formerly smoked occasionally were more likely to participate, whereas current daily smokers and those who abstain from alcohol were less likely to take part.These findings resemble those of the UK Biobank 24 , in that in both studies, participants were less likely to be current smokers and never drinkers compared to nonrespondents from the general population.This pattern has also been seen in a phone-based Stockholm Health of the Population Study 48 .
Taken together, those who self-selected to Hjernetrim constitute only a small part of the general population and are not fully representative of the general population with regard to several sociodemographic attributes, somatic and mental health, cognition, and lifestyle factors.The relatively low participation rate was partly caused by the fact that only those who participated in the main HUNT4 study were invited and by regular mail.Still, the rate was comparable to other general population studies such as the UK Biobank 24 .Among the largest and expected sources of bias were age, educational attainment, self-reported health and certain health conditions such as impaired motor ability.However, contrary to the popular "healthy user bias" 7,[49][50][51] , certain conditions such as cancer and self-reported mental health problems actually increased participation.Moreover, most of the assessed factors, including life-altering ones such as vision and hearing impairment, had relatively small to no effect on participation.Lastly, it is important to note that while self-selection bias is a widespread concern in health research, it does not necessarily compromise the generalizability of the results, as long as its sources are clearly understood and factored into the interpretation of the study's findings 7,44,52 .It can also be accounted for by using certain statistical approaches, for example by using inverse probability weighting 53 .

Hjernetrim engagement
Computer familiarity, hardware, and software In computerized studies such as Hjernetrim, computer familiarity influences participation 54,55 .Our results confirm that the vast majority of those who self-select to web-based studies were comfortable or very comfortable with using computers.This was expected since 95% of people in Norway use the internet daily or almost every day, including more than 75% of those ages 65-79 (ssb.no/en/statbank/table/11124).However, computer familiarity was lower in women and decreased with age, which could have affected the results 56 .Furthermore, since computer familiarity was high in people younger than the late seventies, this issue was limited.An increase in computer familiarity is expected in the future, and a Dutch study reported that less than 5% of older participants did not use a computer frequently 57 .A Malaysian study found that 77% of the sample of older adults had computer-related expertise and only 3% of the sample had none 58 .Similarly, internet usage in Norway has consistently increased among people of all age groups over the last decades.For example, the percentage of individuals aged 55-74 who reported using the internet in the past 3 months surged from 47.8% in 2005 to 99.1% in 2023 (stats.oecd.org/Index.aspx?DataSetCode = ICT_HH2).Overall, computer familiarity seems not to be an issue for most participants and will likely become even less of a concern in the future.
As recommended, most (74%) of the participants used PCs (Windows PC or Macbook) to partake in Hjernetrim.Among those who did not follow this advice, younger participants predominantly chose phones, whereas older participants were more inclined to use tablets.This highlights the need for a web-based platform to be accessible and optimized for all devices to prevent the loss of potential participants.We cannot rule out that we lost participants who did not or could not participate on a computer.Importantly, performing the tests on a phone was associated with a lower rate of completed tasks and a higher likelihood of discontinuation compared to a Windows PC.Participants will experience more disruptions using their mobile devices as the web-based format does not hinder pop-ups or calls.Additionally, conducting tests on smaller screens might present more challenges.Considering that mobile devices are the most commonly used platform for internet access in general (perficient.com/insights/research-hub/mobile-vs-desktop-usage;gs.statcounter.com/platform-market-share/desktop-mobile-tablet) and become the preferred computing device across various settings 59,60 , it is crucial to implement methods for obtaining cognitive data of the same quality irrespective of the type of device.
The Blink browser engine, integral to the recommended Google Chrome browser, emerged as the most popular choice among all age groups, again implying that participants generally adhered to the guidelines.The selection of a browser is primarily a concern for the participant's security 61,62 ; thus, educating participants about this aspect could enhance compliance.However, there may be reluctance among some participants, particularly older adults, to download or use an unfamiliar browser.Our results support this as the use of older and non-standard browsers increased with age.A hesitancy to update the browser could potentially impact participation rates.

Day and time of participation
The preferred time of participation appeared to depend on the convenience of a certain time, i.e., finding a day and time when it was possible to devote ≥ 40 min to taking part in the online testing, and when the different age groups feel their mental capacity peaks at different times during the day, morning for older adults, later in the day for teenagers, and evenings for working age participants.Importantly, while participation at a preferred time does not affect the results in all age groups, it can help older adults perform to the best of their ability 63,64 .Web-based methodology is optimal in this respect.Interestingly, there was a significant variation in participation rates throughout the week.Thursdays and Tuesdays emerged as the most preferred days for participation, while Saturdays and Sundays saw the lowest engagement across almost all age groups.This finding is crucial for planning future web-based study promotions, indicating that advertising efforts might be more effective if concentrated on the weekdays.This conclusion is inconsistent with some previous studies on the impact of invitation day on response rate to online questionnaires 65,66 , but aligns with insights from studies conducted by marketing and survey platforms, as evidenced by the data and recommendations shared on their respective blogs (such as checkmarket.com/blog/survey-invitations-best-time-send,getresponse.com/blog/best-timeto-send-email-infographic,zendesk.com/blog/maximize-survey-response-rates,mailchimp.com/resources/insights-from-mailchimps-send-time-optimization-system).

Time spent
The average time participants spent completing the tests was approximately 46 min, exceeding the 40-min duration outlined in the informational leaflet.This extended duration can be partially attributed to the high performance of participants 46,67 .For example, successfully recalling more words in a verbal memory test results in a longer completion time, while failing a training task could result in skipping the main part of that task.The fact that most participants completed the majority or all of the tasks and provided predominantly positive feedback suggests that a duration of over 40 min is generally acceptable.Yet, it is notable that some individuals, particularly older adults, spent considerably more time than expected, which can lead to participant fatigue, even if it does not decrease performance 67,68 .Therefore, future studies should aim to strike a balance between the length of the tests and the number of cognitive domains assessed, possibly across multiple sessions, to optimize both participation rates and completion rates without inducing fatigue.

Completion and discontinuation of test battery
Overall, the completion rate for all tests was high for Hjernetrim participants.It is important to note that participants who completed fewer tests or discontinued the battery shared some demographic characteristics with those who were less likely to participate in the study overall, such as lacking tertiary education and, in the case of those who completed fewer tests, being in the oldest age groups.This suggests that successfully recruiting harderto-reach individuals for web-based studies does not guarantee complete data.A similar pattern was observed in a previous web-based depression intervention, where lower education levels increased the risk of participants dropping out before completing most of the study modules.This risk was further heightened by younger age and being male 69 .However, we did not observe this in our study, possibly due to differences in the study topics.Additionally, longer or more demanding tasks, such as verbal list learning, appeared to cause relatively high dropout rates.This finding is consistent with previous research that demonstrated an increase in participant discontinuation with longer survey lengths in several web-based psychological questionnaire studies involving undergraduate students 70 .Other verbal memory tests such as verbal paired associates might be a better choice.www.nature.com/scientificreports/Participation at field stations Field stations turned out not to be very popular.Less than 0.5% of all participants participated at a Hjernetrim field station despite advertisements in newspapers, shopping malls, and other HUNT field stations.The participants who chose field stations were predominantly older and not very comfortable with computers, which was most likely the reason they signed up.In theory, field stations could help to recruit important, underrepresented groups, but we were not able to harness such potential.Other solutions, like having the field stations available at the same time as the main HUNT4 data collection was performed, collaborations with senior centres, or help over a remote connection, might have been more efficient.

User experience and feedback for participants
Approximately 28% of participants reported disruptions during one or more tasks.Despite this, the average rate of disruption per task was approximately 5%, which we consider relatively low.This suggests that testing participants in non-controlled environments likely had a limited impact on results, a conclusion supported by previous studies suggesting that web-based methodology does not negatively impact the quality of data and study validity or reliability 3,[71][72][73] .
A positive experience was reported by approximately 65% of participants.Negative experience or technical issues were reported by less than 3% of participants, despite the use of a wide range of devices and software.This suggests that most participants found web-based cognitive testing quite enjoyable, in line with previous Hjernetrim study findings 3 .A caveat here is that participants who provided feedback were mainly those completing all tasks.
Finally, most participants did not log back in for feedback on their performance.A potentially more rewarding and effective approach for feedback could be to provide immediate results or send the results via regular mail like the other results in HUNT4.

Conclusion
This study is the first to examine the characteristics of individuals who volunteered to participate in web-based cognitive testing versus those invited who did not participate and to identify sources of participation bias while also showing in detail participants' engagement in the study.Our study found that groups underrepresented in prior research, such as individuals with mental health concerns, subjective memory complaints, the unemployed and those living alone, were adequately represented in our sample.Differences in factors like educational attainment, age, and various health conditions contributed to varying selection bias.By recognizing and addressing these sources of bias, we can ensure the representativeness and reliability of our future research results.Moreover, the level of participant engagement observed in our study indicates that web-based cognitive testing is effective, even when the test battery is extensive and time-consuming.Finally, the majority of participants reported having a positive experience with the testing process.The patterns of participant engagement can help refine planning, design, and recruitment strategies for future studies that utilize web-based technologies, ultimately boosting participation rates, data quality and participation experience. https://doi.org/10.1038/s41598-024-65617-w

Figure 1 .
Figure 1.(a) Date of participation (year-month) in HUNT4 Hjernetrim (frequency = n).(b) Age and sex distribution of the Hjernetrim participants.

Figure 2 .
Figure 2. Flowchart showing the number of Hjernetrim participants with the number of people invited to the substudy and entire cohorts.

Figure 6 .Figure 7 .
Figure 6.Distribution of devices (a) and browser engines (b) used to log on to the web-based platform, subdivided by age group.

Figure 8 .
Figure 8.Time spent in Hjernetrim in (a) those who completed all tasks and (b) regardless of battery completion.The Y axis illustrates the participant density of each age group. https://doi.org/10.1038/s41598-024-65617-w

Table 1 .
Description of the variables of interest and in which cohorts they were obtained.
YH4: "Do you have impairments of your functioning in any of the following areas?"Motor Impairment, Vision Impairment, Hearing Impairment NT4 and ST1: "If yes [longstanding illness that impairs your functioning], would you describe your impairment as slight, moderate or severe?": 1.Not impaired; 2. Slightly impaired; 3. Moderately impaired; 4. Severely impaired Health YH4, NT4, ST1 Ordinal, 4 values "How is your health at the moment?": 1. Poor; 2.Not so good; 3. Good; 4. Very good Learning disabilities: any, math, reading/writing, other YH4 Nominal, no/yes "Do you have any learning disabilities?If yes, which disabilities?" "Do you have problems with your memory?": 1. No; 2. Yes, some; 3. Yes, a lot "How satisfied are you with your lifestyle (diet, exercise, smoking and drinking habits)?" 1. Very satisfied; 2. Satisfied; 3. Less satisfied;

Table 2 .
Sociodemographic and health characteristics for those participating in the HUNT4 study who did not participate in Hjernetrim and those participating in both the HUNT4 study and Hjernetrim.Results are provided as numbers and percentages.a Measured at the invitation to Memoro.b Hospital Anxiety and Depression Scale Total Score.c Computer familiarity was measured in Memoro participants only.

Table 3 .
Results of linear regression examining the association between sex, education, age group, device and computer familiarity on the number of completed tests.The mean number of completed tasks in Hjernetrim by device and age group.Frequencies below 5 and observations of unidentified hardware were not included in the figure.

Table 4 .
1verview of the user experiences reported by participants at the end of testing stratified into five different topic groups.Results are provided as total numbers and percentages.See materials and methods for more on topic of stratification.1Out of those who left no comments, 1526 (64.7%) did not finish all the tasks.See materials and methods for more on topic stratification.